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IMAGE PROCESSING METHOD AND IMAGE PROCESSING APPARATUS 



5 TECHNICAL FIELD 

The present invention relates to image processing tech- 
nology utilizing resolution conversion for a video signal 
that has been encoded by a high-efficiency compression coding 
method complying with the MPEG-2, for example. 

10 

BACKGROUND ART 

In recent years, the MPEG-2 (Moving Picture Experts 
Group-2 ) standard has been widely used as a high-efficiency 
compression coding method applicable to transmitting or re- 

15 cording a video signal. According to the MPEG-2 standard, 
video frames (or fields) are encoded by classifying them into 
the three picture types of I-, P- and B-pictures. Intra- 
frame coding is performed on an I -picture. Inter-frame for- 
ward predictive coding is performed on a P-picture by using an 

20 I- or P-picture, which is ahead of a current picture on the 
time axis, as a reference frame. And inter-frame bidirection- 
ally predictive coding is performed on a B-picture by using I- 
and/or P-pictures, which are ahead of and behind the current 
picture on the time axis, as respective reference frames. 

25 Motion compensation is utilized in predictively coding P- 
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and B-pictures . According to the MEPG-2, motion compensation 
is performed on a so-called "macroblock" (consisting of 16X16 
pixels) basis. In general, a block matching technique, by 
which a block with the highest correlation is found in a ref- 
5 erence frame, is used in estimating a motion vector for the 
motion compensation purposes. 

Hereinafter, known image processing techniques utilizing 
resolution conversion will be described* 

FIG. 22 is a block diagram for a known image processing 

10 apparatus that performs MPEG-2 decoding, resolution conversion 
and MPEG-2 encoding. In FIG. 22, an encoded stream CSA is de- 
coded by an MPEG-2 decoder 510 to be an original video signal 
SHR with a high resolution. The original video signal SHR is 
converted by a resolution converter 520 into a new video sig- 

15 nal SLR with a low resolution. The new video signal SLR is 
encoded by an MPEG-2 encoder 530 in compliance with the MEPG-2 
so as to be output as an encoded stream CSB. 

If the original video signal SHR is an interlaced signal, 
then the resolution converter 520 usually performs the conver- 

20 sion on a field-by-field basis. This is because if the reso- 
lution conversion is performed on a frame structure as a 
whole, then the resolution will increase in a still-picture 
portion of the frame as compared to the resolution performed 
on a field structure, but a moving-picture portion thereof 

25 cannot be converted appropriately. 
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FIG. 23 illustrates how resolution conversion may be per- 
formed on a pair of fields without changing the structure 
thereof* As shown in FIG. 23, a first field of a new video 
signal is generated from a first field of an original video 

5 signal , and a second field of the new video signal is generat- 
ed from a second field of the original video signal. 

According to another proposed method, motion estimation 
is carried out before resolution conversion to find still- and 
moving -picture portions and then the resolution conversion is 

10 performed on the still- and moving-picture portions using the 
frame and field structures, respectively. 

FIG. 24 illustrates two methods of resolution conversion. 
In FIG. 24, (a) illustrates an image corresponding to an 
original video signal with a high resolution; (b) illustrates 

15 a letterbox image with a low resolution obtained by converting 
the original video signal shown in (a); and (c) illustrates a 
squeeze image with a low resolution obtained by converting the 
original video signal shown in (a). In FIG. 24(a), the aspect 
ratio of the original video signal is supposed to be 16:9. 

20 The aspect ratio of the new video signals is supposed to be 
4:3. As for the letterbox image shown in FIG. 24(b), th« en- 
tire original image has been downscaled at the same ratio both 
vertically and horizontally and data in the shape of black 
bands has been added to the top and bottom of the image. As 

25 for the squeeze image shown in FIG. 24(c) on the other hand, 
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the original image has been downscaled at a ratio vertically 
and at a different ratio horizontally. That is to say, the 
aspect ratio of the image shown in FIG, 24(c) is 4:3. 

5 Problems to be Solved 

The prior art described above, however, has the follow- 
ing drawbacks. 

Firstly, if motion estimation is performed before reso- 
lution conversion, then a huge amount of processing is needed 

10 for the motion estimation, thus increasing the amount of 
hardware or software required. 

Secondly, as for the configuration shown in FIG. 22, the 
MPEG-2 encoder 530 should perform normal MPEG-2 encoding 
processing completely to generate the encoded stream CSB that 

15 is just different in resolution from the encoded stream CSA. 
Thus, the amount of processing will also increase, so the 
amount of hardware or software required rises, too. 

Furthermore, no matter whether the new video signal SLR 
should be represented as a letterbox image or squeeze image, 

20 the MPEG-2 encoder 530 always applies the same encoding 
method thereto and cannot make full use of the characteris- 
tics of a particular resolution conversion method. 

DISCLOSURE OF INVENTION 

25 An object of the present invention is realizing image 
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processing utilizing resolution conversion with the amount of 
processing reduced. 

Specifically, an inventive image processing method in- 
cludes: a decoding step, in which an encoded stream, obtained 

5 by encoding an original video signal with a first resolution, 
is decoded and in which an encoding parameter is extracted 
from the encoded stream; and a resolution converting step, in 
which a characteristic of the original video signal is identi- 
fied by the encoding parameter and in which the original video 

10 signal decoded is converted into a new video signal with a 
second resolution by a resolution conversion method associated 
with the characteristic. 

And in the resolution converting step, a motion charac- 
teristic of a picture represented by the original video signal 

15 is preferably identified as the characteristic of the original 
video signal* Furthermore, the encoding parameter preferably 
includes at least one of: a motion vector representing a quan- 
tity of motion of a video unit; a type of orthogonal transfor- 
mation to be performed using either a frame structure or a 

20 field structure; and a mode of motion compensation to be per- 
formed using either the frame structure or the field struc- 
ture. 

Also, in the resolution converting step, a picture rep- 
resented by the original video signal decoded is preferably 
25 divided into a still area and a moving area by using the en- 
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coding parameter, and the original video signal is preferably 
converted into the new video signal with mutually different 
resolution conversion methods applied to the still and moving 
areas. And the original video signal is preferably an inter- 

5 laced signal* The resolution conversion is preferably per- 
formed on the still area on a frame basis, while the 
resolution conversion is preferably performed on the moving 
area on a field basis. Furthermore, the encoding parameter 
is preferably a motion vector representing a quantity of mo- 

10 tion of a video unit, and the area division is preferably 
carried out based on a result of comparison between the abso- 
lute value of the motion vector and a predetermined value. 

An inventive image processing apparatus includes: a 
video decoder for decoding an encoded stream, obtained by en- 

15 coding an original video signal with a first resolution, and 
for extracting an encoding parameter from the encoded stream; 
and a resolution converter, which receives the original video 
signal and the encoding parameter that have been output from 
the video decoder, identifies a characteristic of the original 

20 video signal by the encoding parameter and converts the origi- 
nal video signal into a new video signal with a second resolu- 
tion by a resolution conversion method associated with the 
characteristic . 

And the resolution converter preferably identifies a mo- 

25 tion characteristic of a picture represented by the original 
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video signal as the characteristic of the original video sig- 
nal- Furthermore, the encoding parameter preferably includes 
at least one of: a motion vector representing a quantity of 
motion of a video unit; a type of orthogonal transformation to 

5 be performed using either a frame structure or a field struc- 
ture; and a mode of motion compensation to be performed using 
either the frame structure or the field structure. 

Also, the resolution converter preferably includes: an 
area dividing section for dividing a picture represented by 

10 the input original video signal into a still area and a mov- 
ing area by using the encoding parameter; a still-area reso- 
lution converting section for converting a video signal 
representing the still area, which has been output from the 
area dividing section, into the video signal with the second 

15 resolution; and a moving-area resolution converting section 
for converting a video signal representing the moving area, 
which has been output from the area dividing section, into 
the video signal with the second resolution- Furthermore, 
the original video signal is preferably an interlaced signal, 

20 the still-area resolution converting section preferably per- 
forms the resolution conversion on a frame basis, and the 
moving-area resolution converting section preferably performs 
the resolution conversion on a field basis. Furthermore, the 
encoding parameter is preferably a motion vector representing 

25 a quantity of motion of a video unit, and the area dividing 
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section preferably performs the area division based on a re- 
sult of comparison between the absolute value of the motion 
vector and a predetermined value. 

Another inventive image processing method includes: a 
5 decoding step, in which an encoded stream , obtained by encod- 
ing an original video signal with a first resolution, is de- 
coded and in which a motion vector is extracted from the 
encoded stream; and a resolution converting step, in which the 
original video signal decoded is converted into a new video 
10 signal with a second resolution by using the motion vector ex- 
tracted. 

And the resolution converting step preferably includes 
an area dividing step, in which a picture represented by the 
original video signal decoded is divided into a quasi-still 

15 area and a moving area by using the motion vector extracted* 
The resolution is preferably converted into that of the new 
video signal by using the extracted motion vector for the 
quasi-still area and without using the extracted motion vector 
for the moving area. 

20 And in the area dividing step, pixel-by-pixel motion 

vectors with directions similar to that of the extracted mo- 
tion vector are preferably estimated from the extracted mo- 
tion vector. An area, in which the pixel-by-pixel motion 
vectors have been estimated, is preferably regarded as the 

25 quasi-still area, while an area, in which none of the vectors 



has been estimated , is preferably regarded as the moving 
area* Furthermore, in the quasi-still area, the resolution 
is preferably converted into that of the new video signal by 
using the pixel-by-pixel motion vectors estimated. 

5 Also, in the area dividing step, an area, in which the 

absolute value of the extracted motion vector is less than a 
predetermined threshold value, is preferably regarded as the 
quasi-still area. On the other hand, an area, in which the 
absolute value is greater than the threshold value, is pref- 

10 erably regarded as the moving area. 

Another inventive image processing apparatus includes: a 
video decoder for decoding an encoded stream, obtained by en- 
coding an original video signal with a first resolution, and 
for extracting a motion vector from the encoded stream; and a 

15 resolution converter, which receives the original video signal 
and the motion vector that have been output from the video de- 
coder and converts the original video signal into a new video 
signal with a second resolution by using the motion vector. 

And the resolution converter preferably includes: an 

20 area dividing section, which receives the original video sig- 
nal and the motion vector and divides a picture represented by 
the original video signal into a quasi-still area and a moving 
area by using the motion vector; a quasi-still-area resolution 
converting section for converting a video signal representing 

25 the quasi-still area, which has been output from the area di- 



9 



viding section, into the video signal with the second resolu- 
tion by using the motion vector; and a moving-area resolution 
converting section for converting a video signal representing 
the moving area, which has been output from the area dividing 

5 section, into the video signal with the second resolution 
without using the motion vector. 

Also, the area dividing section preferably includes a 
motion vector estimating section for estimating, from the mo- 
tion vector, pixel-by-pixel motion vectors with directions 

10 similar to that of the motion vector. An area, in which the 
pixel-by-pixel motion vectors have been estimated by the mo- 
tion vector estimating section, is preferably regarded as the 
quasi-still area, while an area, in which none of the vectors 
has been estimated, is preferably regarded as the moving 

15 area- Furthermore, the quasi-still-area resolution convert- 
ing section preferably converts the resolution by using the 
pixel-by-pixel motion vectors estimated by the motion vector 
estimating section. 

Moreover, the area dividing section preferably regards 

20 an area, in which the absolute value of the motion vector is 
less than a predetermined threshold value, as the quasi-still 
area, and an area, in which the absolute value is greater 
than the threshold value, as the moving area. 

Still another inventive image processing method includes 

25 the steps of; decoding a first encoded stream, obtained by en- 
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coding an original video signal with a first resolution, and 
extracting a first encoding parameter from the first encoded 
stream; converting the original video signal decoded into a 
new video signal with a second resolution; changing the first 

5 encoding parameter into a second encoding parameter for use in 
encoding the new video signal; and encoding the new video sig- 
nal using the second encoding parameter, thereby generating a 
second encoded stream* 

And in the step of changing the encoding parameters, the 

10 first encoding parameter, which has been used to encode a 
first area of a picture represented by the original video 
signal, is preferably changed into the second encoding parame- 
ter, which will be used to encode a second area of a picture 
represented by the new video signal . The second area prefera- 

15 bly includes part of the picture represented in the first 
area. 

And the first and second encoding parameters are prefer- 
ably motion vectors. Furthermore, in the step of changing 
the encoding parameters, a value {e.g., a weighted average), 
20 which has been obtained by performing a predetermined arith- 
metic operation on the motion vectors of the first area, is 
preferably regarded as the motion vector of the second area. 

Also, the first and second encoding parameters prefera- 
bly each represent a type of orthogonal transformation to be 
25 performed using either a frame structure or a field struc- 
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ture. 

Still another inventive image processing apparatus in- 
cludes: a video decoder for decoding a first encoded stream, 
obtained by encoding an original video signal with a first 

5 resolution, and for outputting a first encoding parameter from 
the first encoded stream; a resolution converter for convert- 
ing the original video signal, which has been output from the 
video decoder, into a new video signal with a second resolu- 
tion; an encoding parameter changer for changing the first en- 

10 coding parameter, which has been output from the video 
decoder, into a second encoding parameter for use in encoding 
the new video signal; and a video encoder for encoding the new 
video signal output from the resolution converter by using the 
second encoding parameter output from the encoding parameter 

15 changer, thereby generating a second encoded stream. 

And the encoding parameter changer preferably changes 
the first encoding parameter, which has been used to encode a 
first area of a picture represented by the original video 
signal, into the second encoding parameter, which will be used 

20 to encode a second area of a picture represented by the new 
video signal. The second area preferably includes part of the 
picture represented in the first area. 

And the first and second encoding parameters are prefer- 
ably motion vectors. Furthermore, the encoding parameter 

25 changer preferably regards a value (e.g., a weighted average) 
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which has been obtained by performing a predetermined arith- 
metic operation on the motion vectors of the first area, as 
the motion vector of the second area. 

Also, the first and second encoding parameters prefera- 
5 bly each represent a type of orthogonal transformation to be 
performed using either a frame structure or a field struc- 
ture. 

Yet another inventive image processing method includes 
the steps of: decoding a first encoded stream, obtained by en- 

10 coding an original video signal with a first resolution, and 
extracting a first motion vector from the first encoded 
stream; converting the original video signal decoded into a 
new video signal with a second resolution; defining setting 
information based on the first motion vector to estimate a 

15 second motion vector for use in encoding the new video signal; 
estimating the second motion vector in accordance with the 
setting information defined; and encoding the new video signal 
using the second motion vector estimated, thereby generating a 
second encoded stream. 

20 And an initial value of the second motion vector is 

preferably determined as the setting information* Alterna- 
tively, a search range in which the second motion vector is 
estimated is preferably determined as the setting informa- 
tion. 

25 Yet another inventive image processing apparatus in- 
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eludes: a video decoder for decoding a first encoded stream, 
obtained by encoding an original video signal with a first 
resolution, and for extracting a first motion vector from the 
first encoded stream; a resolution converter for converting 
5 the original video signal, which has been output from the 
video decoder, into a new video signal with a second resolu- 
tion; a motion compensation setter for generating setting in- 
formation based on the first motion vector, which has been 
output from the video decoder, to estimate a second motion 
10 vector for use in encoding the new video signal; and a video 
encoder for estimating the second motion vector in accordance 
with the setting information generated by the motion compensa- 
tion setter and encoding the new video signal, which has been 
output from the resolution converter, using the second motion 
15 vector estimated, thereby generating a second encoded stream. 

And the motion compensation setter preferably determines 
an initial value of the second motion vector as the setting 
information. Alternatively, the motion compensation setter 
preferably determines a search range, in which the second mo- 
20 tion vector is estimated, as the setting information* 

Yet another inventive image processing method includes: 
converting an original video signal having a first resolution 
into a new video signal having a second resolution and a 
black-level area in part of its picture; encoding the new 
25 video signal except the black-level area thereof, thereby 
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generating a first encoded stream; and combining a second en- 
coded stream, obtained by encoding the black-level area of the 
video signal, with the first encoded stream, thereby generat- 
ing an encoded stream for the new video signal, 

5 An alternative inventive image processing method in- 

cludes: converting an original video signal having a first 
resolution into part of a new video signal, the new video 
signal having a second resolution and a black-level area in 
the other part of its picture; encoding the video signal, 

10 thereby generating a first encoded stream; and combining a 
second encoded stream, obtained by encoding the black-level 
area of the video signal, with the first encoded stream, 
thereby generating an encoded stream for the new video signal. 
Yet another inventive image processing apparatus in- 

15 eludes: a resolution converter for converting an original 
video signal having a first resolution into a new video sig- 
nal having a second resolution and a black-level area in part 
of its picture; and a video encoder for encoding the new 
video signal except the black-level area thereof to generate 

20 a first encoded stream and combining a second encoded stream, 
obtained by encoding the black- level area of the video signal, 
with the first encoded stream, thereby generating an encoded 
stream for the new video signal. 

An alternative inventive image processing apparatus in- 

25 eludes: a resolution converter for converting an original 
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video signal having a first resolution into part of a new 
video signal, the new video signal having a second resolution 
and a black-level area in the other part of its picture; and 
a video encoder for encoding the video signal to generate a 
5 first encoded stream and combining a second encoded stream , 
obtained by encoding the black- level area of the video signal, 
with the first encoded stream, thereby generating an encoded 
stream for the new video signal. 

10 BRIEF DESCRIPTION OF DRAWINGS 

FIG. 1 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 
first embodiment of the present invention. 

FIG. 2 is a block diagram illustrating a configuration 
15 for the video decoder shown in FIG. 1. 

FIG. 3 is a block diagram illustrating a configuration 
for the resolution converter shown in FIG. 1. 

FIGS. 4(a) and 4(b) are schematic representations illus- 
trating how motion may be recognized in the first embodiment 
20 of the present invention: FIG. 4(a) illustrates a situation 
where a motion vector is used as an encoding parameter; and 
FIG. 4(b) illustrates a situation where a DCT type is us^d as 
an encoding parameter. 

FIGS. 5(a) through 5(c) are schematic representations 
25 illustrating how area division may be carried out according 
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to the first embodiment of the present invention: FIG. 5(a) 
illustrates a result of motion recognition; FIG* 5(b) illus- 
trates a still area; and FIG. 5(c) illustrates moving areas. 

FIGS. 6(a) and 6(b) are schematic representations illus- 
5 trating how resolution conversion may be performed according 
to the first embodiment of the present invention: FIG. 6(a) 
illustrates a resolution conversion using a frame structure 
as it is; and FIG. 6(b) illustrates a resolution conversion 
using field structures as they are. 
10 FIGS. 7(a) through 7(e) are schematic representations 

illustrating how areas may be combined according to the first 
embodiment of the present invention. 

FIG. 8 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 
15 second embodiment of the present invention. 

FIG. 9 is a block diagram illustrating a configuration 
for the resolution converter shown in FI<5. 8. 

FIG. 10 is a schematic representation illustrating how 
the motion vector estimating and quasi-still-area resolution 
20 converting sections shown in FI<3. 9 may operate. 

FIG. 11 is a schematic representation illustrating how 
the moving-area resolution converting section shown in FIG. 9 
may operate. 

FIG. 12 is a block diagram illustrating another exem- 
25 plary configuration for the resolution converter shown in 
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PIG. 8. 

FIG. 13 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 
third embodiment of the present invention. 
5 FIGS. 14(a) through 14(c) are schematic representations 

illustrating how the encoding parameter changer shown in FIG. 
13 may operate. 

FIG. 15 is a block diagram illustrating a configuration 
for the video encoder shown in FIG. 13. 
10 FIG. 16 is a block diagram illustrating an overall con- 

figuration for an image processing apparatus according to a 
fourth embodiment of the present invention. 

FIG. 17 is a block diagram illustrating a configuration 
for the video encoder shown in FIG. 16. 
15 FIGS. 18(a) through 18{c) are schematic representations 

illustrating how the motion vector calculator shown in FIG. 
17 may operate. 

FIG. 19 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 
20 fifth embodiment of the present invention. 

FIGS. 20(a) and 20(b) illustrate an exemplary resolution 
conversion according to the fifth embodiment of the present 
invention. 

FIG. 21 is a block diagram illustrating a configuration 
25 for the video encoder shown in FIG. 19. 
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FIG. 22 is a block diagram illustrating a configuration 
for a known image processing apparatus. 

FIGS. 23(a) and 23(b) are a schematic representation il- 
lustrating how resolution conversion may be performed using 
5 field structures. 

FIGS. 24(a) through 24(c) illustrate two types of reso- 
lution conversion. 

BEST MODE FOR CARRYING OUT THE INVENTION 

10 (Embodiment 1) 

FIG. 1 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 
first embodiment of the present invention. As shown in FIG. 
1, the image processing apparatus according to this embodi- 

15 ment includes video decoder 10 and resolution converter 20. 
An encoded stream CS of an original video signal with a high 
resolution (i.e., a first resolution) is input. In response, 
the video decoder 10 decodes the input encoded stream CS into 
the original video signal SHR. Not just this decoding , the 

20 decoder 10 also extracts an encoding parameter PAR from the 
encoded stream CS. The resolution converter 20 converts the 
decoded original video signal SHR into a new video signal SLR 
with a low resolution (i.e., a second resolution) using the 
encoding parameter PAR and then outputs the new video signal. 

25 The encoded stream CS is herein supposed to have been 
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encoded in compliance with the MPEG-2. Accordingly , the 
video decoder 10 is a decoder complying with the MPEG-2 (Mov- 
ing Picture Experts Group 2) standard. Also, both the origi- 
nal and new video signals SHR and SLR are herein supposed to 
5 be interlaced signals (i.e., interlaced scanning signals). 

FIG. 2 is a diagram illustrating an internal configura- 
tion for the video decoder 10. As shown in FIG. 2, the video 
decoder 10 includes variable-length decoding section 11, in- 
verse quantizing section 12, inverse DCT (discrete cosine 

10 transform) section 13, frame memory 14, system control sec- 
tion 15, adder 16 and switch 17. The variable-length decod- 
ing section 11 also extracts the encoding parameter PAR. 

Hereinafter, it will be described how the video decoder 
10 shown in FIG. 2 operates. The encoded stream CS repre- 

15 senting one frame has been encoded on a macroblock (consist- 
ing of 16 X 16 pixels) basis. And the macroblocks of the 
encoded stream CS are sequentially input to the video decoder 
10 one after another. 

The input encoded stream CS is variable-length decoded 

20 by the variable-length decoding section 11. Part of the out- 
put of the variable-length decoding section 11, which is asso- 
ciated with picture data, is input to the inverse quantizing 
section 12, while the remaining data other than the picture 
data is sent to the system control section 15. The data de- 

25 livered from the variable-length decoding section 11 to the 
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system control section 15 includes picture types, DCT types 
and motion vectors, which are encoding parameters of the en- 
coded stream CS. As used herein, the "picture type" repre- 
sents whether a frame has been intra-frame coded or inter- 

5 frame coded, while the "DCT type" represents whether the 
structure, with which a block has been subjected to the DCT 
process, is a field structure or a frame structure. 

The data that has been input to the inverse quantizing 
section 12 is inversely quantized and then passed to the in- 

10 verse DCT section 13 . The inverse DCT section 13 performs an 
inverse DCT operation on the data* Herein, where the MPEG-2 
standard is complied with, the DCT process is carried out on 
an 8X8 pixel block basis. If the data has been intra-frame 
coded, then the switch 17 is connected to a. On the other 

15 hand, if the data has been inter- frame coded, then the switch 
17 is connected to b. Supposing the encoded stream CS of an 
inter-frame coded frame is now being processed, the switch 17 
will be connected to b. Thus, the output of the inverse DCT 
section 12 is output to the adder 16, 

20 Using the motion vector that has been sent from the 

variable-length decoding section 11 to the system control 
section 15, a reference picture is retrieved from the frame 
memory 14. A frame to be used as the reference picture has 
already been decoded and stored on the frame memory 14* The 

25 data on which the inverse DCT has been performed (i.e., the 
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output of the inverse DCT section 13) and the reference pic- 
ture (i.e., the data that has been read out from the frame 
memory 14) are added together at the adder 16. And the sum is 
stored as a decoded picture on the frame memory 14* In this 

5 manner, the macroblocks are sequentially decoded and stored on 
the frame memory 14 one after another. 

Decoding will also be performed in the same way on the 
succeeding frames and the results will be sequentially stored 
on the frame memory 14. It should be noted that if the frame 

10 has been intra-frame coded, then the switch 17 will be con- 
nected to a so that the output of the inverse DCT section 13 
will be stored on the frame memory 14 as it is* Alterna- 
tively, if the frame has been inter-frame coded, then the 
switch 17 will be connected to b. In that case, the output of 

15 the inverse DCT section 13 and the reference picture will be 
added together by the adder 16 and the sum will be stored on 
the frame memory 14. 

The decoded pictures, which have been stored on the fra- 
me memory 14 this way, are sequentially output as the origi- 

20 nal video signal SHR with time from the frame memory 14. The 
encoding parameter PAR is also output from the system control 
section 15- 

FIG. 3 is a diagram illustrating an internal configura- 
tion for the resolution converter 20* As shown in FIG. 3, 
25 the resolution converter 20 includes area dividing section 
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21 , still-area resolution converting section 23, moving-area 
resolution converting section 24 and area combining section 
25. The area dividing section 21 includes picture dividing 
section 21a and motion recognizing section 21b . 

5 Hereinafter, it will be described how the resolution 

converter 20 shown in FIG- 3 operates. First, the motion 
recognizing section 21b recognizes the motion characteristic 
of the picture, represented by the original video signal SHR, 
using the encoding parameter PAR received. Then, the motion 

10 recognizing section 21b distinguishes still and moving areas 
from each other. 

FIG. 4 illustrates how the motion recognizing section 
21b recognizes motion. In FIG* 4, (a) illustrates an exem- 
plary motion recognition where a motion vector is used as the 

15 encoding parameter PAR, while (b) illustrates another exem- 
plary motion recognition where a DCT type is used as the en- 
coding parameter PAR. 

First, it will be described with reference -to FIG. 4(a) 
how the motion may be recognized where a motion vector is 

20 used as the encoding parameter PAR. In FIG. 4(a), a frame 
picture has been divided into multiple macroblocks, each of 

which is used as a video unit consisting of 16X16 pixels. 
According to the MPEG- 2 standard, a motion vector represents 
the horizontal and vertical displacement of a macroblock at a 
25 step size of 0.5 pixel. In this case, the motion vector is 



23 



compared to a predetermined value and the motion is recog- 
nized based on a result of the comparison. For example, if 
the absolute value of the motion vector is less than the pre- 
determined value, then the macroblock in question is regarded 
5 as belonging to the still area. Conversely, if the absolute 
value is greater than the predetermined value, then the mac- 
roblock in question is regarded as belonging to the moving 
area. In FIG* 4(a), the solid black macroblocks belong to 
the moving areas, while the open macroblocks belong to the 

10 still area. 

Next, it will be described with reference to FIG. 4(b) 
how the motion may be recognized where a DCT type is used as 
the encoding parameter PAR. In FIG. 4(b), a frame picture 
has been divided into multiple blocks, each consisting of 8 x 

15 8 pixels. According to the MPEG-2 standard, DCT is performed 
on each block using either a frame structure or a field 
structure to obtain the smaller sum of differences between 
vertically adjacent pixels. As used herein, the DCT type 
means the type of the structure the DCT operation has been 

20 performed with. In the illustrated example, if the DCT type 
is the frame structure, then the block in question is regard- 
ed as belonging to the still area. Conversely, if the DCT 
type is the field structure, then the block in question is 
regarded as belonging to the moving area. In FIG. 4(b), the 

25 solid black blocks belong to the moving areas, while the open 
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blocks belong to the still area. 

The motion recognizing section 21b performs the motion 
recognition in this manner and passes its result to the pic- 
ture dividing section 21a- In accordance with the result of 

5 motion recognition obtained from the motion recognizing sec- 
tion 21b, the picture dividing section 21a divides a frame 
picture into the still and moving areas . 

FIG. 5 illustrates schematic representations showing an 
exemplary area division. In FIG. 5, (a) illustrates the re- 

10 suit of motion recognition shown in FIG. 4(a); (b) illus- 
trates a still area that has been extracted by the area 
dividing section 21; and (c) illustrates moving areas ex- 
tracted by the area dividing section 21. 

Part of the video data represented by the original video 

15 signal 3HR, which has been classified as the still area, is 
input to the still-area resolution converting section 23. 
The other part of the video data, which has been classified 
as the moving area, is input to the moving-area resolution 
converting section 24. The still- and moving-area resolution 

20 converting sections 23 and 24 perform the resolution conver- 
sion by respective methods associated with the input video 
data. 

FIG. 6 illustrates how the resolution of the original 
video signal SHR may be converted into that of the new video 
25 signal SLR in the still- and moving-area resolution convert- 
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ing sections 23 and 24. In FIG, 6, columns of pixels verti- 
cally arranged are illustrated* Specifically, "O" represents 
a pixel belonging to a first field, while "A" represents a 
pixel belonging to a second field. In the example illustrat- 

5 ed in FIG. 6, the vertical pixels are reduced by 1/2. In FIG. 
6, (a) illustrates a resolution conversion using a frame 
structure as it is, while (b) illustrates a resolution con- 
version using field structures as they are. 

The still-area resolution converting section 23 performs 

10 the resolution conversion using the frame structure as it is. 
Specifically, as shown in FIG* 6(a), the video data of the 
first field for the new video signal SLR is generated by us- 
ing pixels belonging to both the first and second fields of 
the original video signal SHR. The video data of the second 

15 field for the new video signal SLR is also generated by using 
pixels belonging to both the first and second fields of the 
original video signal SHR* 

On the other hand, the moving-area resolution converting 
section 24 performs the resolution conversion using the field 

20 structures as they are. Specifically, as shown in FIG. 6(b), 
the video data of the first field for the new video signal 
SLR is generated only from the pixels belonging to the first 
field of the original video signal SHR. And the video data 
of the second field for the new video signal SLR is generated 

25 only from the pixels belonging to the second field of the 
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original video signal SHR. 

The new video signals, which have been generated by the 
still- and moving-area resolution converting sections 23 and 
24 to represent the still and moving areas, respectively, are 

5 output to the area combining section 25. The area combining 
section 25 combines these new video signals for the still and 
moving areas with each other, thereby converting the resul- 
tant signal into a frame picture and outputting it as a new 
video signal SLR. 

10 FIG. 7 illustrates how the area combining section 25 may 

operate. In FIG. 7, (a) and (b) illustrate the pictures in- 
put to the still- and moving-area resolution converting sec- 
tions 23 and 24, respectively, i.e*, the pictures shown in 
FIGS . 5(b) and 5(c). FIGS. 7(c) and 7(d) illustrate the pic- 

15 tures output from the still- and moving-area resolution con- 
verting sections 23 and 24, respectively. The number of 
pixels has been cut down as a result of the resolution conver- 
sion. And the area combining section 25 produces a combined 
picture such as that illustrated in FIG. 7(e), i.e., the new 

20 video signal SLR, from the pictures shown in FIGS. 7(c) and 
7(d). 

As described above, according to this embodiment, an en- 
coded stream of an original video signal is decoded while an 
encoding parameter is extracted from the encoded stream. 
25 Next, the characteristic of the original video signal is rec- 
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ognized by the encoding parameter and then the original video 
signal is converted into a new video signal by a resolution 
converting method associated with the characteristic . In this 
manner, the amount of processing, including motion vector cal- 
5 culation for resolution conversion purposes, can be reduced 
considerably. In addition, a picture, represented by the 
original video signal, is classified as still and moving areas 
according to the encoding parameter and the resolution conver- 
sion is carried out with mutually different methods applied to 

10 the still and moving areas* As a result, a new video signal 
of quality can be obtained with the amount of processing dras- 
tically reduced. 

In the foregoing embodiment, motion vectors and OCT 
types are used as exemplary encoding parameters PAR to recog- 

15 nize the motion characteristic of a picture. Alternatively, 
any other encoding parameter may also be used. Examples of 
other applicable encoding parameters include motion compensa- 
tion modes* As used herein, the "motion compensation mode" 
represents whether the structure with which motion compensa- 

20 tion has been performed is a field structure or frame struc- 
ture. In using the motion compensation modes as the encoding 
parameters PAR of this embodiment, the resolution conversion 
may be performed using the field structures for an area with a 
motion compensation mode associated with the field structures. 

25 And as for an area with a motion compensation mode associated 
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with the frame structure , the resolution conversion may be 
performed using the frame structure. 

Also, in this embodiment, the resolution conversion is 
performed using the frame structure for the still area and 
5 the field structures for the moving areas, respectively. Al- 
ternatively, any other resolution converting method is also 
applicable. 

Moreover, in performing the resolution conversion using 
the frame or field structures according to this embodiment, 
10 each pixel for the new video signal SLR is generated from the 
values of two vertically adjacent pixels for the original 
video signal SHR as shown in FIGS - 6(a) and 6(b). However, 
any other combination of pixels may be used instead. 

Furthermore, in this embodiment, the resolution is 
15 halved. Alternatively, the resolution may be cut down at any 
other ratio. 

Furthermore, in this embodiment, an original video sig- 
nal with a high resolution is converted into a new video sig- 
nal with a low resolution. However, even if a low-resolution 
20 original video signal should be converted into a high- 
resolution new video signal conversely, the encoding parame- 
ters may also be used as in this embodiment. 

(Embodiment 2) 

25 FIG. 8 is a block diagram illustrating an overall con- 
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figuration for an image processing apparatus according to a 
second embodiment of the present invention. As shown in FIG, 
8, the image processing apparatus according to this embodi- 
ment includes video decoder 110 and resolution converter 120. 
5 An encoded stream CS of an original video signal with a low 
resolution (i.e., a first resolution) is input. In response, 
the video decoder 110 decodes the input encoded stream CS in- 
to the original video signal SLR* Not just this decoding, 
the decoder 110 also extracts a motion vector MV from the en- 

10 coded stream CS. The resolution converter 120 converts the 
decoded original video signal SLR into a new video signal SHR 
with a high resolution (i.e., a second resolution) using the 
motion vector MV and outputs the new video signal. 

The original video signal SLR is herein supposed to be 

15 an interlaced signal (i.e., interlaced scanning signal), 
while the new video signal SHR is herein supposed to be a 
progressive signal (i.e., progressive scanning signal).. 

The video decoder 110 has basically the same configura- 
tion as the video decoder 10 according to the first embodi- 

20 ment. The only difference between these decoders 110 and 10 
is that the decoder 110 outputs the motion vector MV. Thus a 
detailed description thereof will be omitted herein - 

FIG. 9 is a block diagram illustrating an internal con- 
figuration for the resolution converter 120. As shown in 

25 FIG. 9, the resolution converter 120 includes area dividing 
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section 121, quasi-still-area resolution converting section 
123 , moving-area resolution converting section 124, area com- 
bining section 125 and frame memory 127. Also, the area di- 
viding section 121 includes motion vector estimating section 
5 121a, motion recognizing section 121b and picture dividing 
section 121c, 

The frame memory 127 stores the input original video 
signals SLR thereon. Using the motion vector MV output from 
the video decoder 110, a current field of the original video 

10 signal SLR and preceding fields thereof stored on the frame 
memory 127, the motion vector estimating section 121a esti- 
mates a pixel-by-pixel motion vector MVa, which has a direc- 
tion similar to that of the motion vector MV, for the current 
field and another field temporally close to the current 

15 field* 

Hereinafter, it will be described with reference to FIG. 
10 how the motion vector estimating section 121a operates. 
FIG. 10 schematically illustrates pixels of three frames n, 
n+1 and n+2 of the original video signal SLR. In FIG. 10, 

20 represents a pixel belonging to the first field of each 

frame, while * X " represents a pixel belonging to the second 
field of each frame. 

Suppose the first field of the frame n+2 should be con- 
verted into a progressive picture (i.e., progressively scan- 

25 ned picture) with a twice higher resolution. And it will be 
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described how to generate the pixel D (represented by "A") 
shown in FIG. 10. 

In this case, a motion vector A for an area G, including 
the pixel D to be generated, is used among the motion vectors 

5 MV output from the video decoder 110, The motion vector A 
originates from an area F belonging to the first field of the 
frame n. And a motion vector E with the same direction as 
that of the motion vector A is selected from the motion vec- 
tors of the field closest to the pixel D, i.e., the second 

10 field of the frame n+1. 

Then, a more accurate motion vector B for a pixel neigh- 
boring the pixel D is selected from motion vectors with di- 
rections similar to that of the motion vector E by using the 
pixels neighboring the pixel D in the first field of the fra- 

15 me n+2 and the pixels in the second field of the frame n+1- 
That is to say, the pixel D can be regarded as a pixel C that 
has moved from the second field of the frame n+1 by the mo- 
tion vector B. 

The motion vector estimating section 121a outputs the 
20 motion vector B, which has been estimated in this manner, as 
the pixel-by-pixel motion vector MVcr to the motion recogniz- 
ing section 121b. 

Using the motion vector MVa that has been input from 
the motion vector estimating section 121a, the motion recog- 
25 nizing section 121b distinguishes the quasi-still and moving 
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areas from each other. In the illustrated embodiment, pix- 
els, for which no pixel-by-pixel motion vectors MV a have 
been found by the motion vector estimating section 121a, are 
regarded as belonging to the moving area. On the other hand, 

5 pixels, for which the pixel-by-pixel motion vectors MVa have 
been found, are regarded as belonging to the quasi-still 
area. The motion recognizing section 121b performs the mo- 
tion recognition in this manner, and passes the result to the 
picture dividing section 121c. 

10 As used herein, the "quasi-still area' 1 means an area of 

a picture that can be handled as if it were a still picture 
by shifting the picture by the quantity of the motion using 
the pixel-by-pixel motion vector MV a although that area is 
actually a part of the picture with motion. 

15 In accordance with the result of motion recognition ob- 

tained from the motion recognizing section 121b, the picture 
dividing section 121c divides a frame picture into the quasi- 
still and moving areas . Part of the video data that has been 
classified as the quasi-still area is input to the quasi- 

20 still-area resolution converting section 123, while the other 
part of the video data classified as the moving area is input 
to the moving-area resolution converting section 124. The 
quasi-still-area and moving-area resolution converting sec- 
tions 123 and 124 perform resolution conversion on the input 

25 video data by respective methods. 
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Hereinafter, it will be described with reference to FIG. 

10 how the quasi-still-area resolution converting section 123 
perforins the resolution conversion. The quasi-still-area re- 
solution converting section 123 performs the resolution con- 

5 version using the original video signal SLR, the moving area 
picture output from the area dividing section 121 and the 
pixel-by-pixel motion vector MVor output from the motion vec- 
tor estimating section 121a. 

Suppose the first field of the frame n+2 should be sub- 

10 jected to the resolution conversion, for example. In this 
case, since the motion vector B has been estimated as the 
pixel -by-pixel motion vector MVa for a pixel neighboring the 
pixel D, the pixel C belonging to the second field of the 
frame n+1 is retrieved from the frame memory 127 . And the 

15 pixel value thereof is used as the value of the pixel D. In 
the same way, pixels at other interpolation locations are al- 
so generated using the pixel-by-pixel motion vectors MV a 
that have been estimated by the motion vector estimating sec- 
tion 121a. As a result, the high-resolution video signal is 

20 generated ♦ 

Hereinafter, it will be described with reference to FIG. 

11 how the moving-area resolution converting section 124 per- 
forms the resolution conversion. In FIG. 11, "O" represents 
a pixel for the low-resolution video signal, while "X" rep- 

25 resents a pixel generated, by interpolation for the high- 
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resolution video signal. As shown in FIG. 11, the moving- 
area resolution converting section 124 generates the high- 
resolution video signal from the pixels in the field of the 
low-resolution video signal through interpolation . For exam- 

5 pie, the value of a pixel C for the high-resolution video 
signal is the same as that of a pixel A for the low- 
resolution video signal. The value of a pixel D for the 
high-resolution video signal is obtained from those of the 
pixels A and B for the low-resolution video signal. 

10 The video signals, which have been generated by the 

quasi-still-area and moving-area resolution converting sec- 
tions 123 and 124, are input to the area combining section 
125 . The area combining section 125 combines the input video 
signals together, thereby outputting the combined signal as a 

15 new video signal SHR. 

This embodiment includes the motion vector estimating 
section 121a for re-calculating the motion vector MVa on a 
pixel-by-pixel basis using the motion vector MV that has been 
obtained from the encoded stream CS of the original video 

20 signal SLR. Alternatively, the area division may also be car- 
ried out using the value of the estimated motion vector MV as 
it is, 

FIG. 12 is a diagram illustrating a configuration for a 
resolution converter 120A for such an alternative embodiment. 
25 In FIG. 12, the area dividing section 122 includes a motion 
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recognizing section 122a for recognizing the motion based on 
the motion vector MV directly and a picture dividing section 
122b. In this case, the area division may be carried out 
with a pixel regarded as belonging to the quasi-still area if 

5 the absolute value of the motion vector MV is less than a 
predetermined threshold value or as belonging to the moving 
area if the absolute value is greater than the threshold 
value, for example . Also, a pixel may be regarded as belong- 
ing to the moving area only when the absolute value of the mo- 

10 tion vector MV is very great* Furthermore, the resolution 
conversion may be carried out while regarding the entire area 
as the quasi-still area and using the value of the motion vec- 
tor MV as it is. 

As described above, according to this embodiment, an en- 

15 coded stream of an original video signal is decoded while a 
motion vector is extracted from the encoding stream. Then, a 
new video signal is generated through resolution conversion 
using the motion vector extracted. In this manner, the amount 
of processing, including motion vector calculation for resolu- 

20 tion conversion purposes, can be reduced considerably. In ad- 
dition, quasi-still and moving areas of a picture represented 
by the original video signal are distinguished by the motion 
vector and the resolution conversion is carried out on the 
quasi-still area using the motion vector. Thus, the amount of 

25 processing, including motion vector calculation carried out to 
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convert the resolution for the quasi-still area, can be cut 
down significantly. Furthermore, the resolution conversion is 
performed with mutually different methods applied to the 
quasi-still and moving areas. As a result, a new video signal 
5 of quality can be obtained with the amount of processing dras- 
tically reduced. 

In this embodiment, an original video signal with a low 
resolution is converted into a new video signal with a high 
resolution. However, even if a high-resolution original 
10 video signal should be converted into a low-resolution new 
video signal conversely, the motion vectors may also be used 
as in this embodiment. 

Also, in this embodiment, the motion vector section 121a 
estimates a motion vector between a current field and a past 
15 field, and the quasi-still-area resolution converting section 
123 generates an interpolated pixel from pixels belonging to a 
preceding field. Alternatively, a motion vector may also be 
estimated for a future field and an interpolated pixel may al- 
so be generated from pixels belonging to a future field. 
20 Moreover, in this embodiment, the moving-area resolution 

converting section 124 converts the resolution by generating 
interpolated pixels as shown in FIG. 11. Optionally, any 
other resolution conversion method is applicable. 

Furthermore, in this embodiment, the quasi-still-area re- 
25 solution converting section 123 generates an interpolated pix- 
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el from pixels belonging to the closest/ preceding field. Al- 
ternatively/ an interpolated pixel may also be generated from 
pixels belonging to a field that is several fields away from 
the current field. 

5 

( Embodiment 3 ) 

FIG. 13 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 
third embodiment of the present invention. As shown in FIG. 

10 13, the image processing apparatus according to this embodi- 
ment includes video decoder 210, resolution converter 220, 
video encoder 230 and encoding parameter changer 240. 

A first encoded stream CSA, which is an encoded stream 
of an original video signal with a high resolution (i.e., a 

15 first resolution), is input. In response, the video decoder 
210 decodes the input first encoded stream CSA into the 
original video signal SHR. Not just this decoding, the de- 
coder 210 also extracts an encoding parameter PARA from the 
first encoded stream CSA. The resolution converter 220 con- 

20 verts the original video signal SHR into a new video signal 
SLR. The encoding parameter changer 240 changes the first 
encoding parameter PARA, which has been output from the video 
decoder 210, into a parameter for a video with a low resolu- 
tion and then outputs it as a second encoding parameter parb. 

25 And the video encoder 230 encodes the new video signal SLR us- 
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ing the second encoding parameter PARB and then outputs it as 
a second encoded stream CSB. 

The first encoded stream CSA is herein supposed to have- 
been encoded in compliance with the MPEG-2. Accordingly, the 
5 video decoder 210 is a decoder complying with the MPEG-2 
standard. The configuration and operation of the video de- 
coder 210 are the same as those of the video decoder 10 ac- 
cording to the first embodiment. Thus, the detailed 
description thereof will be omitted herein. 
10 The resolution converter 220 converts the original video 

signal SHR, which has been decoded by the video decoder 210, 
into the new video signal SLR. In the illustrated embodi- 
ment, the resolution converter 220 cuts down the number of 
pixels by half both vertically and horizontally. 
15 Next, it will be described with reference to FIG. 14 how 

the encoding parameter changer 240 operates. The encoding 
parameter changer 240 changes the first encoding parameter 
PARA for the original video signal SHR into the second encod- 
ing parameter PARB for the new video signal SLR. Suppose a 
20 frame picture of the original video signal SHR is converted 
into a frame picture of the new video signal SLR that has had 
its number of pixels reduced by half both vertically and hori- 
zontally as shown in FIG. 14(a). 

First, a situation where the encoding parameter to be 
25 changed is a motion vector will be described with reference to 
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FIG- 14(b)- In FIG- 14(b), a frame picture is illustrated as 
having been divided into multiple macroblocks. As shown in 
FIG. 14(b), an area for macroblocks MBH1, MBH2, MBH3 and MBH4 
in the original video signal SHR (i.e., the first area) is 

5 downscaled to an area for a macroblock MBL in the new video 
signal SLR (i.e., the second area) through the resolution con- 
version. In this case, a motion vector MVL for the macroblock 
MBL in the new video signal SLR is supposed to be obtained by 
averaging weighted motion vectors MVHi of the respective mac- 

10 roblocks MBHi in the original video signal SHR. That is to 
say, the motion vector MVL is given by the following Equation 

(1): 

N 

MVL={2 (CiXMVHi)}X a (1) 

15 i = l 

where Ci is a weighting coefficient meeting the following 

Equation ( 2 ) : 
N 

2 Ci=l (2) 
20 i=l 

where N is the number of macroblocks in the original video 
signal SHR that are used to estimate the motion vector MVL for 
the macroblock MBL in the new video signal SLR. In the illus- 
trated example, N is "4". Also, a is a ratio of the number 
25 of pixels representing the picture size of the new video sig- 
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nal to the number of pixels representing the picture size of 
the original video signal. Normally, a horizontal a value is 
different from a vertical a value. 

In this case, if the pixel number ratio resulting from 

5 the resolution conversion is 1/2 or 1/4, for example, then the 
boundary between macroblocks after the conversion matches to 
the boundary between macroblocks before the conversion. Ac- 
cordingly, the same weighting coefficient Ci value can be 
used. However, if the pixel number ratio is 1/3 or 2/5, for 

10 example, then the boundary between macroblocks after the con- 
version does not match to the boundary between macroblocks be- 
fore the conversion. Thus, the macroblocks should be weighted 
by mutually different weighting coefficient Ci values before 
the conversion. 

15 Next, a situation where the encoding parameter to be 

changed is a DCT type (i-e., type of orthogonal transforma- 
tion) will be described with reference to FIG. 14(c). In FIG. 
14(c), a frame picture is illustrated as having been divided 
into multiple blocks. As shown in FIG. 14(c), an area for 

20 blocks BH1, BH2, BH3 and BH4 in the original video signal 
(i.e., the first area) is downscaled to an area for a block BL 
in the new video signal (i.e., the second area) through the 
resolution conversion. In this case, a DCT type for the block 
BL in the new video signal is obtained by using the DCT types 

25 of the respective blocks BH1 through BH4 in the original video 
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signal. For example, if the DCT types of the blocks BHl 
through BH4 are all a frame structure, then the DCT type of 
the block BL may also be the frame structure. Or if at least 
one of the DCT types of the blocks BHl through BH4 is a field 
5 structure, then the DCT type of the block BL should also be 
the field structure. Alternatively, the DCT type of the con- 
verted block may be defined as the DCT type applied to the 
greater number of original blocks* 

In this manner, the encoding parameter changer 240 
10 changes the first encoding parameter PARA for the original 
video signal SHR into the second encoding parameter PARB for 
the new video signal SLR. 

FIG. 15 is a block diagram illustrating an internal con- 
figuration for the video encoder 230. The video encoder 230 
15 shown in FIG. 15 has basically the same configuration as a 
normal MPEG encoder. However, since a motion vector may be 
provided as the second encoding parameter PARB, the motion 
estimating section is omitted therefrom. Also, since a DCT 
type may be provided as the second encoding parameter PARB, 
20 means for determining the DCT type by calculating a variance, 
for example, is omitted from a DCT calculating section 233. 

In the illustrated example, the video encoder 230 is 
supposed to encode the new video signal SLR in compliance 
with the MPEG-2 standard, and is also supposed to perform in- 
25 ter-frame encoding. 
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First, the new video signal SLR, which has been input to 
the video encoder 230, is divided by a blocking section 231 
into multiple macroblocks (each consisting of 16X16 pixels). 
Then, the macroblocks are sequentially input to a motion com- 
5 pensation section 232 one after another. The motion compen- 
sation section 232 performs motion compensation on the input 
macroblocks by using the motion vector included in the second 
encoding parameter PARB that has been provided from the encod- 
ing parameter changer 240. Specifically, the motion compensa- 
10 tion section 232 reads out a reference macroblock from a frame 
memory 238 by using the motion vector and obtains a difference 
between the reference and input macroblocks, thereby perform- 
ing the motion compensation. The resultant differential mac- 
roblock is input to the DCT calculating section 233 • 
15 The DCT calculating section 233 converts the differen- 

tial macroblock into DCT coefficients on an 8X8 pixel block 
basis. In this case, the DCT calculating section 233 per- 
forms the DCT calculation in accordance with the DCT type in- 
cluded in the second encoding parameter PARB that has been 
20 provided from the encoding parameter changer 240. The resul- 
tant DCT coefficients are output to a quantizing section 234, 
which performs a quantization process on the DCT coefficients. 
A variable-length encoding section 235 performs a variable- 
length encoding process on the output of the quantizing sec- 
25 tion 234 and then outputs the result as the second encoded 
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stream CSB. 

Also, the output of the quantizing section 234 is decod- 
ed by an inverse quantizing section 236 and an inverse DCT 
section 237. Subsequently, the decoded data is added to the 
5 reference macroblock, which has been read out from the frame 
memory 238, by an adder 239, and then the sum is stored on the 
frame memory 238. The data stored will be used as a reference 
picture when succeeding frames are encoded. 

As described above, according to this embodiment, a 
10 first encoded stream of an original video signal is decoded 
and the decoded original video signal is converted into a new 
video signal through resolution conversion. Also, in parallel 
with this decoding, a first encoding parameter such as motion 
vector or DCT type is extracted from the first encoded stream. 
15 Next, the first encoding parameter is changed into a second 
encoding parameter for the new video signal, and the new video 
signal is converted into a second encoded stream by using this 
second encoding parameter. Accordingly, there is no need to 
obtain encoding parameters in encoding the new video signal, 
20 thus cutting down the amount of processing considerably. Par- 
ticularly when motion vectors are used as the encoding parame- 
ters, the amount of processing can be reduced very 
significantly. 

In this embodiment, the encoding parameter to be changed 
25 is a motion vector or a DCT type. However, the present inven- 
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tion is implement able similarly even by using any other encod- 
ing parameter such as a mode of motion compensation. 

Also, the encoding parameter changing method of this em- 
bodiment is just an illustrative one. Thus, the conversion 

5 may be carried out by any other method. In this embodiment, 
the encoding parameter for a predetermined area (i.e./ mac- 
roblock or block) of the new video signal is obtained by using 
only the encoding parameter for an area of the original video 
signal that corresponds to the predetermined area. Option- 

10 ally, not just the encoding parameter for the area of the 
original video signal corresponding to the predetermined area, 
but also encoding parameters for other surrounding areas may 
be used. 

Moreover, in this embodiment, the motion vectors are 
15 converted by adopting a weighting and averaging method as a 
predefined arithmetic operation. Examples of other imagi- 
nable and applicable methods include a method of selecting a 
centroid of a motion vector component, a method of neglecting 
small motion vector quantities and a method of weighting using 
20 predicted errors. 

Furthermore, in this embodiment, the same weighting co- 
efficient Ci is used if the macroblock boundaries match be- 
fore and after the conversion. However, the coefficient does 
not always have to be the same. 
25 Furthermore, in this embodiment, an original video sig- 
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nal with a high resolution is converted into a new video sig- 
nal with a low resolution. However, even if a low- resolution 
original video signal should be converted into a high- 
resolution new video signal conversely, the encoding parame- 
5 ters may also be changed as in this embodiment. 

(Embodiment 4) 

FIG. 16 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 

10 fourth embodiment of the present invention. As shown in FIG. 
16, the image processing apparatus according to this embodi- 
ment includes video decoder 310, resolution converter 320, 
video encoder 330 and motion compensation setter 340. 

A first encoded stream CSA, which is an encoded stream 

15 of an original video signal with a high resolution (i.e., a 
first resolution), is input. In response, the video decoder 
310 decodes the input first encoded stream CSA into the 
original video signal SHR. Not just this decoding, the de- 
coder 310 also extracts a motion vector MV from the first en- 

20 coded stream CSA. The resolution converter 320 converts the 
original video signal SHR into a new video signal SLR with a 
low resolution (i.e., a second resolution). The motion com- 
pensation setter 340 defines settings for the motion compen- 
sation, which should be carried out by the video encoder 330, 

25 by using the motion vector MV of the encoded stream CSA that 
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has been output from the video decoder 310. And the video en- 
coder 330 estimates a second motion vector in accordance with 
the setting information SET output from the motion compensa- 
tion setter 340, encodes the new video signal SLR using the 
5 second motion vector and then outputs it as a second encoded 
stream CSB. 

The configurations and operations of the video decoder 
310 and resolution converter 320 are the same as those of the 
video decoder 110 and resolution converter 320 according to 

10 the second embodiment. Thus, the detailed description there- 
of will be omitted herein. 

Hereinafter, it will be described how the motion compen- 
sation setter 340 operates. The motion compensation setter 
340 receives the motion vector MV that has been obtained by 

15 the video decoder 310 for the original video signal SHR, and 
obtains the setting information SET for motion compensation to 
be carried out by the video encoder 330. As the setting in- 
formation SET, an initial value of motion compensation (i.e., 
an initial value of the second motion vector) and a range of 

20 motion compensation (i.e., a search range for the second mo- 
tion vector) are determined, for example. 

First, it will be described with reference to FIG. 14(b) 
how the initial value of motion compensation is obtained as 
the motion compensation setting information SET. In FIG. 

25 14(b), a frame picture is illustrated as having been divided 
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into multiple macroblocks . As shown in FIG, 14(b), an area 
for macroblocks MBH1 , MBH2 , MBH3 and MBH4 in the original 
video signal is downscaled to an area for a macroblock MBL in 
the new video signal through the resolution conversion. In 

5 this case, the motion compensation setter 340 may obtain the 
motion vector MVL by the Equation (1) described above, for ex- 
ample, as a motion compensation initial value that will be 
used by the video encoder 310 in encoding the macroblock MBL. 
And the setter 340 outputs this motion vector MVL as the moti- 

10 on compensation setting information SET. 

Next, it will be described with reference to FIG, 14(b) 
how the range of motion compensation is determined as the mo- 
tion compensation setting information SET . In that case, the 
motion compensation setter 340 determines the motion compensa- 

15 tion range, from which the video encoder 330 will have to es- 
timate a motion vector for the macroblock MBL, by using the 
motion vectors of the macroblocks MBH1 through MBH4. To de- 
fine the motion compensation range, the maximum absolute val- 
ues of positive and negative motion vector components of the 

20 respective macroblocks MVHl through MVH4 may be used according 
to a method, for example. Alternatively, according to another 
method, the average or variance value of the motion vectors of 
the respective macroblocks MVHl through MVH4 may also be used. 
In this manner, the motion compensation setter 340 de- 

25 fines the initial value or range for calculating the motion 
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vector for the new video signal as the motion compensation 
setting information SET by using the motion vector MV of the 
original video signal SHR. Then, the setter 340 outputs the 
value or range to the video encoder 330. 

5 FIG . 17 is a block diagram illustrating an internal con- 

figuration for the video encoder 33. In the illustrated ex- 
ample, the video encoder 330 is supposed to encode the new 
video signal SLR in compliance with the MPEG-2 standard, and 
is also supposed to perform inter-frame encoding. 

10 First, the new video signal SLR, which has been input to 

the video encoder 330, is divided by a blocking section 331 
into multiple macroblocks (each consisting of 16X16 pixels). 
Then, the macroblocks are sequentially input to a motion com- 
pensation section 332 and a motion vector calculator 350 one 

15 after another. 

The motion vector calculator 350 calculates the second 
motion vector MV2 for each of the input macroblocks in accor- 
dance with the motion compensation setting information SET 
that has been provided from the motion compensation setter 

20 340. At this stage, a reference picture is retrieved from a 
frame memory 338. 

Hereinafter, .it will be described with reference to FIG. 
18 how the motion vector calculator 350 operates. In FIG. 
18, (a) illustrates a frame to be encoded, which is repre- 

25 sented by the new video signal SLR and has been input from 
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the resolution converter 320 , while (b) and (c) illustrate 
the reference frames that have been read out from the frame 
memory 338. Suppose a motion vector should be estimated for 
the macroblock MBl shown in FIG. 18(a). 

5 First, a situation where an initial value of motion com- 

pensation has been input as the motion compensation setting 
information SET will be described with reference to FIG. 
18(b). In this case, the motion vector calculator 350 uses 
the initial value SET, which has been obtained by the motion 

10 compensation setter 340, as the initial value of the second 
motion vector MV2. And an area surrounding the endpoint of 
the motion vector SET originating from a macroblock MBl a , 
which is located at the same position as the macroblock MBl, 
is defined as a search range SRI in which the second motion 

15 vector MV2 should be estimated. In this search range SRI, the 
second motion vector MV2 is calculated for the macroblock MBl. 

Next, a situation where a range of motion compensation 
has been input as the motion compensation setting information 
SET will be described with reference to FIG. 18(c). In that 

20 case, the motion vector calculator 350 uses the range SET, 
which has been obtained by the motion compensation setter 340, 
as the search range SR2 in which the second motion vector MV2 
should be estimated. And in this search range SR2 (i.e., the 
range SET), the second motion vector MV2 is calculated for the 

25 macroblock MBl. 
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The motion compensation section 332 performs motion com- 
pensation on the input macroblocks by using the second motion 
vector MV2 provided from the motion vector calculator 350* 
Specifically, the motion compensation section 332 reads out a 
5 reference macroblock from the frame memory 338 by using the 
second motion vector MV2 and obtains a difference between the 
reference and input macroblocks, thereby performing the motion 
compensation* The resultant differential macroblock is input 
m to a DCT calculating section 333* 

T] 10 The DCT calculating section 333, quantizing section 334, 

In ^ variable-length encoding section 335, inverse quantizing sec- 
||I tion 336, inverse DCT calculating section 337 and adder 339 

b operate in the same way as the counterparts of the second em- 

?V bodiment. Thus, the description thereof will be omitted 

'^i 15 herein. The second encoded stream CSB is output from the 
W variable- length encoding section 335. 

As described above, according to this embodiment, a 
first encoded stream of an original video signal is decoded 
and the decoded original video signal is converted into a new 
20 video signal through resolution conversion. Also, in parallel 
with this decoding, a first motion vector that has been used 
for encoding the original video signal is extracted. Next, 
motion compensation setting information such as the initial 
value and range is defined to estimate a second motion vector, 
25 which will be used for encoding the new video signal, from the 
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first motion vector- And in accordance with this setting in- 
formation, the second motion vector is estimated f thereby en- 
coding the new video signal. As a result, the search range 
can be much narrowed compared to the prior art, and the amount 

5 of motion vector computation can be considerably reduced in 
encoding a new video signal. Also, since the motion compensa- 
tion setting information can be obtained from the first motion 
vector that has been used for encoding the original video sig- 
nal, the accuracy of motion compensation can be kept high. 

10 Also, the method of defining the motion compensation 

setting information according to this embodiment is just an 
illustrative one. Thus, any other method may be used. For 
example, not just the motion vector for a macroblock of the 
original video signal corresponding to a predetermined mac* 

15 roblock of the new video signal, but also motion vectors for 
other macroblocks surrounding the former macroblock may be 
used. Moreover, in this embodiment, the initial value of the 
second motion vector is determined by adopting a weighting 
and averaging method. Examples of other imaginable and ap- 

20 plicable methods include a method of selecting a centroid of 
a motion vector component, a method of neglecting small motion 
vector quantities and a method of weighting using predicted 
errors . 

Furthermore, the motion compensation setter 340 may de- 
25 fine both the initial value and search range of the second 
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motion vector as the motion compensation setting information 
SET. 

Furthermore, in this embodiment, an original video sig- 
nal with a high resolution is converted into a new video sig- 
5 nal with a low resolution* However, even if a low-resolution 
original video signal should be converted into a high- 
resolution new video signal conversely, the motion compensa- 
tion setting information SET may also be defined as in the 
present invention. 

10 

(Embodiment 5) 

FIG. 19 is a block diagram illustrating an overall con- 
figuration for an image processing apparatus according to a 
fifth embodiment of the present invention- As shown in FIG. 

15 19, the image coding apparatus according to this embodiment 
includes video decoder 410, resolution converter 420, video 
encoder 430 and system control section 440. 

An encoded stream CSA of a video signal with a high 
resolution {i.e., a first resolution) is input. In response, 

20 the video decoder 410 decodes the input encoded stream CSA 
into the original video signal SHR. The resolution converter 
420 converts the decoded original video signal SHR into a new 
video signal SLR with a low resolution (i.e., a second reso- 
lution). The video encoder 430 encodes the new video signal 

25 SLR and then outputs it as a second encoded stream CSB. 
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The first encoded stream CSA is herein supposed to have 
been encoded in compliance with the MPEG-2. Accordingly, the 
video decoder 410 is a decoder complying with the MPEG-2 
standard. The configuration and operation of the video de- 

5 coder 410 are the same as those of the video decoder 10 ac- 
cording to the first embodiment- Thus, the detailed 
description thereof will be omitted herein* 

As shown in FIG. 20, the resolution converter 420 con- 
verts the original video signal SHR into a video signal rep- 

10 resenting a letterbox picture. In the example illustrated in 
FIG. 20, the original video signal SHR with an aspect ratio 
of 16:9 as shown in (a) is converted into a new video signal 
SLR with an aspect ratio of 4:3 as shown in (b) . If the pic- 
ture has been converted into the letterbox picture, black- 

15 level areas 451 and 452 in the shape of bands are added to 
the frame picture 450 of the new video signal SLR over and 
under the picture. 

FIG. 21 is a block diagram illustrating an internal con- 
figuration for the video encoder 430. As shown in FIG. 21, 

20 the video encoder 430 includes area dividing section 431, 
blocking section 432, DCT calculating section 433, quantizing 
section 434, variable-length encoding section 435, encoded 
stream generating section 436 and encoded stream storage sec- 
tion 437. In the illustrated embodiment, the video encoder 

25 430 is supposed to encode the new video signal SLR by the 
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MPEG-2 and is also supposed to perform intra-frame encoding. 

On receiving an instruction signal SI from the system 
control section 440 , the area dividing section 431 extracts 
and outputs an effective data area (i.e., the area 450 except 

5 the black-level areas 451 and 452 shown in FIG. 20(b)) from 
the letterbox picture represented by the new video signal 
SLR. That is to say, the video signal representing the 
black-level areas 451 and 452 is not output. The blocking 
section 432 divides the video signal representing the input 

10 area 450 into multiple blocks. The DCT calculating section 
433 converts the blocked video signal into DCT coefficients. 
The DCT coefficients, output from the DCT calculating section 
433, are quantized by the quantizing section 434, converted 
by the variable-length encoding section 435 into a first en- 

15 coded stream CSBl and then output to the encoded stream gen- 
erating section 436. 

On the encoded stream storage section 437, second encod- 
ed streams CSB2 , which have been obtained by encoding the 
video signal representing the black-level areas 451 and 452, 

20 are stored in advance. Both intra-frame and inter-frame en- 
coded streams are stored as the second encoded streams CSB2 
on the encoded stream storage section 437 . In the illustrat- 
ed embodiment, one of the second encoded streams CSB2 that 
has been intra-frame encoded is output from the encoded 

25 stream storage section 437. 
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The encoded stream generating section 436 combines the 
first encoded stream CSB1 output from the variable-length en- 
coding section 435 with the second encoded stream CSB2 output 
from the encoded stream storage section 437. In the illus- 

5 trated embodiment, these encoded streams are combined sequen- 
tially by beginning with the top of the frame. That is to 
say, the encoded stream representing the black-level area 451 
output from the encoded stream storage section 437, the encod- 
ed stream representing the area 450 output from the variable- 

10 length encoding section 435 and the encoded stream represent- 
ing the black-level area 452 output from the encoded stream 
storage section 437 are combined together in this order. The 
encoded stream combined in this manner is output as the encod- 
ed stream CSB. 

15 It should be noted that the resolution converter 420 may 

output the new video signal SLR with the black-level areas 
451 and 452 eliminated. In that case, the area dividing sec- 
tion 431 may be omitted from the video encoder 430. 

Also, the area dividing section 431 may determine wheth- 

20 er the input new video signal SLR represents a letterbox pic- 
ture or not. In that case, there is no need to externally 
provide the instruction signal SI to the video encoder 430. 

As described above, according to this embodiment, where 
a new video signal with a letterbox structure, obtained from 

25 an original video signal through resolution conversion, is en- 
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coded, no encoding is actually performed on black-level areas 
thereof but pre-stored encoded streams are combined. Accord- 
ingly, there is no need to perform encoding processing on the 
black-level areas, thus considerably cutting down on the 

5 amount of processing* For example, suppose a frame represent- 
ed by the new video signal has a size of 720 horizontal pixels 
by 480 vertical pixels and the area other than the black-level 
areas has a vertical size of 360 pixels. In that case, the 
amount of processing can be reduced by 25% compared to encod- 

10 ing the entire picture. 

In this embodiment, the black-level areas are added over 
and under the effective data area. Alternatively, at least 
one black-level area may be added either over or under the da- 
ta area. Also, the black-level area{s) may be added to vari- 

15 ous parts other than the upper and lower parts, e.g., on 
right- and left-hand sides, on the left side only or on the 
right side only. 

Moreover, in this embodiment, the original video signal 
with an aspect ratio of 16:9 is converted into the new video 

20 signal with an aspect ratio of 4:3. However, aspect ratios in 
any other combination may be used instead. 

Furthermore, in this embodiment, an original video sig- 
nal with a high resolution is converted into a new video sig- 
nal with a low resolution* However, even if a low-resolution 

25 original video signal should be converted into a high- 
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resolution new video signal, encoding of the black-level areas 
can also be omitted as in the present invention. 

Furthermore , in this embodiment , the video encoder 430 
performs intra-frame encoding. But the same statements are 
5 applicable to a situation where the video encoder 430 performs 
inter- frame encoding. 

Furthermore, in the foregoing embodiments, the MPEG- 2 is 
used as an exemplary encoding method. Alternatively, any oth- 
er encoding method like the MPEG-1 or the H.261 is also appli- 
10 cable. 
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CLAIMS 



1. An image processing method characterized by compris- 
ing: 

a decoding step, in which an encoded stream , obtained by 
encoding an original video signal with a first resolution, is 
decoded and in which an encoding parameter is extracted from 
the encoded stream; and 

a resolution converting step, in which a characteristic 
of the original video signal is identified by the encoding 
parameter and in which the original video signal decoded is 
converted into a new video signal with a second resolution by 
a resolution conversion method associated with the character- 
istic. 

2. The image processing method of Claim 1, characterized 
in that in the resolution converting step, a motion charac- 
teristic of a picture represented by the original video signal 
is identified as the characteristic of the original video sig- 
nal, 

3. The image processing method of Claim 2, characterized 
in that the encoding parameter comprises at least one of: a 
motion vector representing a quantity of motion of a video 
unit; a type of orthogonal transformation to be performed us- 
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ing either a frame structure or a field structure; and a mode 
of motion compensation to be performed using either the frame 
structure or the field structure. 

4. The image processing method of Claim 1, characterized 
in that in the resolution converting step, 

a picture represented by the original video signal de- 
coded is divided into a still area and a moving area by using 
the encoding parameter, and 

the original video signal is converted into the new 
video signal with mutually different resolution conversion 
methods applied to the still and moving areas. 

5. The image processing method of Claim 4, characterized 
in that the original video signal is an interlaced signal, 
and 

that the resolution conversion is performed on the still 
area on a frame basis, while the resolution conversion is 
performed on the moving area on a field basis. 

6. The image processing method of Claim 4, characterized 
in that the encoding parameter is a motion vector represent- 
ing a quantity of motion of a video unit, and 

that the area division is carried out based on a result 
of comparison between the absolute value of the motion vector 
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and a predetermined value. 



7. The image processing method of Claim 1, characterized 
in that the encoded stream of the original video signal has 
been encoded in compliance with an MPEG (Moving Picture Ex- 
perts Group) standard. 

80 The image processing method of Claim 1, characterized 
in that the first resolution is higher than the second reso- 
lution. 

9* An image processing apparatus characterized by com- 
prising: 

a video decoder for decoding an encoded stream, obtained 
by encoding an original video signal with a first resolution, 
and for extracting an encoding parameter from the encoded 
stream; and 

a resolution converter, which receives the original video 
signal and the encoding parameter that have been output from 
the video decoder, identifies a characteristic of the original 
video signal by the encoding parameter and converts the origi- 
nal video signal into a new video signal with a second resolu- 
tion by a resolution conversion method associated with the 
characteristic . 
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10. The image processing apparatus of Claim 9, charac- 
terized in that the resolution converter identifies a motion 
characteristic of a picture represented by the original video 
signal as the characteristic of the original video signal. 

11. The image processing apparatus of Claim 10 , charac- 
terized in that the encoding parameter comprises at least one 
of: a motion vector representing a quantity of motion of a 
video unit; a type of orthogonal transformation to be per- 
formed using either a frame structure or a field structure; 
and a mode of motion compensation to be performed using either 
the frame structure or the field structure. 

12. The image processing apparatus of Claim 9, charac- 
terized in that the resolution converter comprises: 

an area dividing section for dividing a picture repre- 
sented by the input original video signal into a still area 
and a moving area by using the encoding parameter; 

a still-area resolution converting section for convert- 
ing a video signal representing the still area, which has 
been output from the area dividing section, into the video 
signal with the second resolution; and 

a moving-area resolution converting section for convert- 
ing a video signal representing the moving area, which has 
been output from the area dividing section, into the video 
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signal with the second resolution. 

13. The image processing apparatus of Claim 12, charac- 
terized in that the original video signal is an interlaced 
signal, and 

that the still-area resolution converting section per- 
forms the resolution conversion on a frame basis, and 

that the moving-area resolution converting section per- 
forms the resolution conversion on a field basis. 

14. The image processing apparatus of Claim 12, charac- 
terized in that the encoding parameter is a motion vector 
representing a quantity of motion of a video unit, and 

that the area dividing section performs the area divi- 
sion based on a result of comparison between the absolute 
value of the motion vector and a predetermined value. 

15. The image processing apparatus of Claim 9, charac- 
terized in that the encoded stream of the original video sig- 
nal has been encoded in compliance with an MPEG standard. 

16. The image processing apparatus of Claim 9, charac- 
terized in that the first resolution is higher than the sec- 
ond resolution. 
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17. An image processing method characterized by compris- 
ing: 

a decoding step, in which an encoded stream, obtained by 
encoding an original video signal with a first resolution, is 
decoded and in which a motion vector is extracted from the en- 
coded stream; and 

a resolution converting step, in which the original video 
signal decoded is converted into a new video signal with a 
second resolution by using the motion vector extracted. 

18. The image processing method of Claim 17, character- 
ized in that the resolution converting step comprises 

an area dividing step in which a picture represented by 
the original video signal decoded is divided into a quasi- 
still area and a moving area by using the motion vector ex- 
tracted, and 

that the resolution is converted into that of the new 
video signal by using the extracted motion vector for the 
quasi-still area and without using the extracted motion vector 
for the moving area. 

19. The image processing method of Claim 18, character- 
ized in that in the area dividing step, 

pixel-by-pixel motion vectors with directions similar to 
that of the extracted motion vector are estimated from the 
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extracted motion vector, and 

an area, in which the pixel-by -pixel motion vectors have 
been estimated, is regarded as the quasi-still area, while an 
area, in which none of the vectors has been estimated, is re- 
garded as the moving area, 

20. The image processing method of Claim 19, character- 
ized in that in the quasi-still area, the resolution is con- 
verted into that of the new video signal by using the pixel- 
by-pixel motion vectors estimated. 

21. The image processing method of Claim 18, character- 
ized in that in the area dividing step, 

an area, in which the absolute value of the extracted 
motion vector is less than a predetermined threshold value, 
is regarded as the quasi-still area, while an area, in which 
the absolute value is greater than the threshold value, is 
regarded as the moving area. 

22. The image processing method of Claim 17, character- 
ized in that the encoded stream of the original video signal 
has been encoded in compliance with an MPEG standard. 

23. The image processing method of Claim 17, character- 
ized in that the first resolution is lower than the second 
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resolution. 

24. An image processing apparatus characterized by com- 
prising: 

a video decoder for decoding an encoded stream, obtained 
by encoding an original video signal with a first resolution , 
and for extracting a motion vector from the encoded stream; 
and 

a resolution converter, which receives the original video 
signal and the motion vector that have been output from the 
video decoder and converts the original video signal into a 
new video signal with a second resolution by using the motion 
vector . 

25. The image processing apparatus of Claim 24, charac- 
terized in that the resolution converter comprises: 

an area dividing section, which receives the original 
video signal and the motion vector and divides a picture rep- 
resented by the original video signal into a quasi-still area 
and a moving area by using the motion vector; 

a quasi-still-area resolution converting section for con- 
verting a video signal representing the quasi-still area, 
which has been output from the area dividing section, into the 
video signal with the second resolution by using the motion 
vector; and 
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a moving-area resolution converting section for convert- 
ing a video signal representing the moving area, which has 
been output from the area dividing section, into the video 
signal with the second resolution without using the motion 
vector . 

26. The image processing apparatus of Claim 25, charac- 
terized in that the area dividing section comprises 

a motion vector estimating section for estimating, from 
the motion vector, pixel-by-pixel motion vectors with direc- 
tions similar to that of the motion vector, and 

that an area, in which the pixel-by-pixel motion vectors 
have been estimated by the motion vector estimating section, 
is regarded as the quasi-still area, while an area, in which 
none of the vectors has been estimated, is regarded as the 
moving area. 

27. The image processing apparatus of Claim 26, charac- 
terized in that the quasi-still-area resolution converting 
section converts the resolution by using the pixel-by-pixel 
motion vectors estimated by the motion vector estimating sec- 
tion. 

28. The image processing apparatus of Claim 25, charac- 
terized in that the area dividing section regards an area, in 
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which the absolute value of the motion vector is less than a 
predetermined threshold value, as the quasi-still area, and 
also regards an area, in which the absolute value is greater 
than the threshold value, as the moving area, 

29. The image processing apparatus of Claim 24, charac- 
terized in that the encoded stream of the original video sig- 
nal has been encoded in compliance with an MPEG standard* 

30. The image processing apparatus of Claim 24, charac- 
terized in that the first resolution is lower than the second 
resolution. 

31. An image processing method characterized by compris- 
ing the steps of: 

decoding a first encoded stream, obtained by encoding an 
original video signal with a first resolution, and extracting 
a first encoding parameter from the first encoded stream; 

converting the original video signal decoded into a new 
video signal with a second resolution; 

changing the first encoding parameter into a second en- 
coding parameter for use in encoding the new video signal; and 

encoding the new video signal using the second encoding 
parameter, thereby generating a second encoded stream. 
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32. The image processing method of Claim 31, character- 
ized in that in the step of changing the encoding parameters, 
the first encoding parameter, which has been used to encode a 
first area of a picture represented by the original video 
signal, is changed into the second encoding parameter, which 
will be used to encode a second area of a picture represented 
by the new video signal, the second area including part of the 
picture represented in the first area. 

33* The image processing method of Claim 32, character- 
ized in that the first and second encoding parameters are mo- 
tion vectors* 

34. The image processing method of Claim 33, character- 
ized in that in the step of changing the encoding parameters, 
a value, which has been obtained by performing a predeter- 
mined arithmetic operation on the motion vector of the first 
area, is regarded as the motion vector of the second area. 

35. The image processing method of Claim 33, character- 
ized in that in the step of changing the encoding parameters, 
a weighted average of the motion vectors of the first area is 
regarded as the motion vector of the second area. 

36. The image processing method of Claim 32, character- 
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ized in that the first and second encoding parameters each 
represent a type of orthogonal transformation to be performed 
using either a frame structure or a field structure, 

37. The image processing method of Claim 31, character- 
ized in that the first encoded stream has been encoded in 
compliance with an MPEG standard. 

38. The image processing method of Claim 31, character- 
ized in that the second encoded stream is encoded in compli- 
ance with an MPEG standard. 

39. An image processing apparatus characterized by com- 
pris ing: 

a video decoder for decoding a first encoded stream, ob- 
tained by encoding an original video signal with a first reso- 
lution, and for outputting a first encoding parameter from the 
first encoded stream; 

a resolution converter for converting the original video 
signal, which has been output from the video decoder, into a 
new video signal with a second resolution; 

an encoding parameter changer for changing the first en- 
coding parameter, which has been output from the video de- 
coder, into a second encoding parameter for use in encoding 
the new video signal; and 
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a video encoder for encoding the new video signal output 
from the resolution converter by using the second encoding 
parameter output from the encoding parameter changer, thereby 
generating a second encoded stream. 

40. The image processing apparatus of Claim 39, charac- 
terized in that the encoding parameter changer changes the 
first encoding parameter, which has been used to encode a 
first area of a picture represented by the original video 
signal, into the second encoding parameter, which will be used 
to encode a second area of a picture represented by the new 
video signal, the second area including part of the picture 
represented in the first area. 

41. The image processing apparatus of Claim 40, charac- 
terized in that the first and second encoding parameters are 
motion vectors . 

42. The image processing apparatus of Claim 41, charac- 
terized in that the encoding parameter changer regards a 
value, which has been obtained by performing a predetermined 
arithmetic operation on the motion vector of the first area, 
as the motion vector of the second area. 

43. The image processing apparatus of Claim 41, charac- 
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terized in that the encoding parameter changer regards a 
weighted average of the motion vectors of the first area as 
the motion vector of the second area. 

44. The image processing apparatus of Claim 40, charac- 
terized in that the first and second encoding parameters each 
represent a type of orthogonal transformation to be performed 
using either a frame structure or a field structure. 

45. The image processing apparatus of Claim 39 , charac- 
terized in that the first encoded stream has been encoded in 
compliance with an MPEG standard. 

46. The image processing apparatus of Claim 39, charac- 
terized in that the second encoded stream is encoded in com- 
pliance with an MPEG standard. 

47. An image processing method characterized by compris- 
ing the steps of: 

decoding a first encoded stream, obtained by encoding an 
original video signal with a first resolution, and extracting 
a first motion vector from the first encoded stream; 

converting the original video signal decoded into a new 
video signal with a second resolution; 

defining setting information based on the first motion 
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vector to estimate a second motion vector for use in encoding 
the new video signal; 

estimating the second motion vector in accordance with 
the setting information defined; and 

encoding the new video signal using the second motion 
vector estimated, thereby generating a second encoded stream, 

48. The image processing method of Claim 47, character- 
ized in that an initial value of the second motion vector is 
determined as the setting information. 

49. The image processing method of Claim 47, character- 
ized in that a search range in which the second motion vector 
is estimated is determined as the setting information. 

50. The image processing method of Claim 47, character- 
ized in that the first encoded stream has been encoded in 
compliance with an MPEG standard. 

51. The image processing method of Claim 47, character- 
ized in that the second encoded stream is encoded in compli- 
ance with an MPEG standard. 

52. An image processing apparatus characterized by com- 
prising: 
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a video decoder for decoding a first encoded stream, ob- 
tained by encoding an original video signal with a first 
resolution, and for extracting a first motion vector from the 
first encoded stream; 

a resolution converter for converting the original video 
signal, which has been output from the video decoder, into a 
new video signal with a second resolution; 

a motion compensation setter for generating setting in- 
formation based on the first motion vector, which has been 
output from the video decoder, to estimate a second motion 
vector for use in encoding the new video signal; and 

a video encoder for estimating the second motion vector 
in accordance with the setting information generated by the 
motion compensation setter and encoding the new video signal, 
which has been output from the resolution converter, using the 
second motion vector estimated, thereby generating a second 
encoded stream. 

53. The image processing apparatus of Claim 52, charac- 
terized in that the motion compensation setter determines an 
initial value of the second motion vector as the setting in- 
formation. 

54. The image processing apparatus of Claim 52, charac- 
terized in that the motion compensation setter determines a 



74 



search range, in which the second motion vector is estimated , 
as the setting information. 

55. The image processing apparatus of Claim 52, charac- 
terized in that the first encoded stream has been encoded in 
compliance with an MPEG standard. 

56. The image processing apparatus of Claim 52 , charac- 
terized in that the second encoded stream is encoded in com- 
pliance with an MPEG standard. 

57. An image processing method characterized by: 
converting an original video signal having a first reso- 
lution into a new video signal having a second resolution and 
a black-level area in part of its picture; 

encoding the new video signal except the black-level 
area thereof, thereby generating a first encoded stream; and 

combining a second encoded stream, obtained by encoding 
the black-level area of the video signal, with the first en- 
coded stream, thereby generating an encoded stream for the new 
video signal. 

58. An image processing method characterized by: 
converting an original video signal having a first reso- 
lution into part of a new video signal, the new video signal 
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having a second resolution and a black-level area in the oth- 
er part of its picture; 

encoding the video signal , thereby generating a first 
encoded stream; and 

combining a second encoded stream, obtained by encoding 
the black-level area of the video signal , with the first en- 
coded stream , thereby generating an encoded stream for the new 
video signal. 

59. The image processing method of Claim 57 or 58, char- 
acterized in that the encoding is performed in compliance 
with an MPEG standard. 

60. An image processing apparatus characterized by com- 
prising: 

a resolution converter for converting an original video 
signal having a first resolution into a new video signal hav- 
ing a second resolution and a black-level area in part of its 
picture; and 

a video encoder for encoding the new video signal except 
the black-level area thereof to generate a first encoded 
stream and combining a second encoded stream, obtained by en- 
coding the black-level area of the video signal, with the 
first encoded stream, thereby generating an encoded stream for 
the new video signal. 
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61. An image processing apparatus characterized by com- 
prising: 

a resolution converter for converting an original video 
signal having a first resolution into part of a new video 
signal, the new video signal having a second resolution and a 
black-level area in the other part of its picture; and 

a video encoder for encoding the video signal to gener- 
ate a first encoded stream and combining a second encoded 
stream, obtained by encoding the black-level area of the video 
signal, with the first encoded stream, thereby generating an 
encoded stream for the new video signal. 

62. The image processing apparatus of Claim 60 or 61, 
characterized in that the video encoder performs the encoding 
in compliance with an MPEG-2 standard. 
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ABSTRACT 

A video decoder 10 not only decodes an encoded stream CS 
to generate an original video signal SHR with a first resolu- 
tion but also extracts an encoding parameter PAR such as mo- 
5 tion vector or type of orthogonal transformation from the 
encoded stream CS. A resolution converter 20 recognizes a 
characteristic of the original video signal SHR by the encod- 
ing parameter PAR, and converts the original video signal SHR 
into a new video signal SLR with a second resolution by a re- 
10 solution conversion method associated with the characteristic* 
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